1. Field of the Invention
This invention relates generally to communication systems, and, more particularly, to transmission of multimedia data streams in communication systems.
2. Description of the Related Art
Digital multimedia has become a viable alternative to conventional analog transmission of video and/or audio data. For example, packet-based techniques such as Internet Protocol television (IPTV) may be used to provide digital multimedia services such as television, video-on-demand, and the like. In a typical digital multimedia broadcast system, a sequence of images is converted into a digital representation of the images. The digital data is then encoded and transmitted to a receiving unit, such as a television or a computer, which includes a buffer for storing the received information for several images before providing the encoded information to a decoder for decoding and eventual presentation of the image. The decoder buffer may overflow if the transmitter attempts to send too much digital data to the receiving unit, resulting in data loss and possible picture drop. Conversely, the decoder buffer may underflow if the transmitter does not send enough digital data to the receiving unit. Underflow may cause a decoder stall and the repetition of one or more images. Since overflow and underflow of the decoder buffer may degrade user experience, conventional digital multimedia systems vary the encoding quality of each image to affect bit rate to attempt to maintain the decoder buffer level within a selected range.
Service providers may want to splice together multiple data streams to broadcast additional information to users. For example, a service provider may want to splice encoded advertising information into encoded multimedia content that the service provider is broadcasting or multicasting to one or more users in an encoded data stream. One splicing technique is to decode the multimedia data stream and the advertising data stream. The decoded data streams may then be spliced together and then re-encoded for transmission to the receiving unit. This approach has the advantage that the information in the spliced data streams is encoded by a single encoder at a bit rate that is selected to maintain the decoder buffer level within the selected range. Buffer overflow and/or underflow may then be avoided by selecting an appropriate bit rate. However, this approach requires additional hardware and/or software to decode the multiple data streams. For example, a splicer may include one decoder for each data stream, a video switch for splicing the two decoded data streams, and an encoder for re-encoding the spliced data stream.
One alternative approach is to simply splice together the encoded data streams at a selected splice point. Although this approach requires less hardware and/or software than techniques that splice together decoded data streams, simply splicing together encoded data streams may lead to decoder buffer underflow and/or overflow. For example, the encoded data streams may have been separately encoded using different models of the decode buffer. The decode buffer models typically assume an initial decode buffer level that is transmitted periodically from the encoder to the decoder within the multimedia stream to allow decoders to adjust their decode buffer status appropriately when they begin to decode the stream. However, in practice, the decode buffer level at the splice point will differ randomly and unpredictably from the assumed initial decode buffer level because it will not typically be possible to predict the decode buffer level resulting from decoding of the first data stream.
FIGS. 1A and 1B conceptually illustrate a decode buffer level 100 as a function of time for an embodiment in which two data streams are spliced together. In the illustrated embodiment, the decode buffer level increases upward and time increases from left to right. A buffer underflow occurs when the decode buffer level 100 drops below the minimum buffer threshold 105 and a buffer overflow occurs when the decode buffer level 100 rises above the maximum buffer threshold 110. The splice point between the first and second data streams is indicated by the line 115. The bit rate for the first data stream is determined based upon a first model of the decode buffer and the fullness of the decode buffer, which is determined by the relative rate at which information arrives at buffer and is subsequently played out. Accordingly, the decode buffer does not underflow or overflow prior to the splice point 115.
In FIG. 1A, the splice point 115 occurs when the decode buffer level 100 is lower than the initial decode buffer level assumed by a second model that was used to encode the second data stream. The bit rate used to encode the second data stream is determined based upon the second model and the size (or fullness) of the decode buffer. However, since the assumed initial decode buffer level is higher than the actual initial decode buffer level, the encoder bit rate used to encode the second data stream is too low, which results in underflow of the decode buffer at the times 120, 125, 130.
In FIG. 1B, the splice point 115 occurs when the decode buffer level 100 is higher than the initial decode buffer level assumed by the second model that was used to encode the second data stream. The encoder bit rate used to encode the second data stream is therefore too high because the assumed initial decode buffer level is higher than the actual initial decode buffer level. Adopting the relatively high encoder bit rate results in too much information being transmitted, which causes an overflow of the decode buffer at the times 135, 140, 145.
FIG. 2 conceptually illustrates a first embodiment of a timing diagram 200 that illustrates splicing together two data streams. Time increases from the left to the right in FIG. 2. In the illustrated embodiment, the timing diagram 200 depicts compressed frames 205 of encoded digital information and uncompressed frames 210 that are formed from the encoded digital information in the packets 205 and then presented to users. The arrows 215 indicate the association between the compressed frame packets 205 and the uncompressed frames 210 formed using these packets 205. In the interest of clarity, only one packet, frame, and arrow are indicated by numerals 205, 210, 215, respectively. A first encoded data stream 220, such as a multimedia data stream used to present audio and/or video to a user, is provided and may be spliced with a second encoded data stream 225 such as multimedia advertising content that a service provider may want to display to the user during the audio and/or video presentation.
In the illustrated embodiment, a spliced data stream 230 is formed by concatenating the first encoded data stream 220 and the second encoded data stream 225. The concatenation occurs at a splice point 235 defined by the end of the last packet in the first encoded data stream 220. Encoding of the second encoded data stream 225 does not take into account the state of the decoder buffer at the splice point 235. Consequently, the packet 240 arrives too late to be used to form the frame 245, as indicated by the backwards-facing arrow 250. This condition is another representation of an underflow at the decoder buffer, i.e. the decoder buffer has played out all of its contents prior to arrival of the packet 240 and so there is no information available to be decoded and used to form the frame 245. The frame 245 may therefore be a duplicate of a previous frame or an incompletely formed frame.
FIG. 3 conceptually illustrates a second embodiment of a timing diagram 300 that illustrates splicing together two data streams. Time increases from the left to the right in FIG. 3. In the illustrated embodiment, the timing diagram 300 depicts packets 305 of encoded digital information and the frames 310 that are formed from the encoded digital information in the packets 305 and then presented to users. The arrows 315 indicate the association between the packets 305 and the frames 310 formed using these packets 305. In the interest of clarity, only one packet, frame, and arrow are indicated by numerals 305, 310, 315, respectively. A first encoded data stream 320, such as a multimedia data stream used to present audio and/or video to a user, is provided and may be spliced with a second encoded data stream 325 such as multimedia advertising content that a service provider may want to display to the user during the audio and/or video presentation.
In the illustrated embodiment, a spliced data stream 330 is formed by concatenating the first encoded data stream 320 and the second encoded data stream 325. The concatenation occurs at a splice point 335 defined by the end of the last packet in the first encoded data stream 320. Encoding of the second encoded data stream 325 does not take into account the state of the decoder buffer at the splice point 335 and the fullness of the decoder buffer is larger than expected. The packet 340 arrives too early to be used immediately (or within a selected delay time) to form the frame 345, as indicated by the extended arrow 350. This condition is another representation of an overflow at the decoder buffer, i.e. the decoder buffer has to store the information included in the packet 340 for an unexpectedly long time. Subsequently received packets may therefore fill the decoder buffer before the information in the packet 340 can be read out and decoded. Information in the decoder buffer may then be lost as newly received information is rejected or previously received information is popped out of the decoder buffer.